Optimal Classification with Multivariate Losses

A. Appendix A A.1. Proof of Theorem 1 The proof is by contradiction. Fix a distribution P satisfying conditional independence, and let x denote a fixed set of instances. Denote P(Y = 1|xi) = ηi and the optimal classifier by s∗ ∈ {0, 1}n. Suppose there exist indices j, k such that sj = 1, sk = 0 and ηj < ηk. Let s′ ∈ {0, 1}n be such that sj = 0 and sk = 1, but identical to s∗ otherwise i.e. si = si ∀i ∈ [n]\{j, k}. Note that ∑n i=1 s ∗ i = ∑n i=1 s ′ i. For convenience, define: U(s;P) := EY∼P(.|x)L(s,Y) .